Otoscanner With 3D Imaging And Onboard Tracking

ABSTRACT

Apparatus and methods for tracking scanner motion with a tracking illumination emitter mounted on a scanner, the scanner including imaging apparatus that captures a sequence of images of a scanned object as the scanner moves with respect to the scanned object; tracking targets installed off the scanner at positions that are fixed relative to a scanned object; one or more tracking illumination sensors mounted on the scanner, the tracking illumination sensors sensing reflections of tracking illumination emitted from the tracking illumination emitter and reflected from the tracking targets as the scanner moves in the process of scanning the scanned object; and one or more data processors, at least one of the data processors coupled for data communications to the tracking illumination sensors, the data processor determining, as the scanner moves, tracked positions of the scanner based upon values of the reflections.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of and claims priorityfrom U.S. patent application Ser. No. 13/417,649, filed on Mar. 12,2012.

BACKGROUND OF THE INVENTION

The present invention relates to determining the shape of surfaces withparticular reference to surfaces of soft tissue, and more specifically,to determining such shapes using optical technology. Hearing aids,hearing protection, and custom head phones often require siliconeimpressions to be made of a patient's ear canal. Audiologists inject thesilicone material into an ear, wait for it to harden, and then providethe mold to manufacturers who use the resulting silicone impression tocreate a custom fitting in-ear device. The process is slow, expensive,inconsistent, unpleasant for the patient, and can even be dangerous, asinjecting silicone risks affecting the ear drum. Also, there are a rangeof other medical needs that benefit from determining the shape of bodysurfaces, including surfaces defining body orifices, such as the size ofshape of an ear canal, throat, mouth, nostrils, or intestines of apatient. For example, surgery may be guided by knowing such shapes, andmedical devices may be fashioned to have a custom fit for such shapes.

Example embodiments are described in terms of otoscanners withparticular reference to scanning ears. This is for explanation and notfor limitation. In fact, readers will recognize that scanners accordingto embodiments of the present invention may be used to scan almost any3D surface of live or inanimate objects both internal and external.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A sets forth a line drawing of an example otoscanner.

FIGS. 1B-1E set forth line drawings of further example otoscanners.

FIG. 2 sets forth a line drawing of an even further example otoscanner.

FIGS. 3A and 3B illustrate projections of laser light onto surfaces of ascanned ear.

FIG. 4 sets forth a flow chart illustrating an example method ofconstructing a 3D image of a scanned ear.

FIG. 5 sets forth a line drawing illustrating additional examplefeatures of an ear probe and image sensor of an otoscanner according toembodiments of the present invention.

FIG. 6 sets forth a line drawing of an example ear probe (106) of anotoscanner according to embodiments of the present invention.

FIGS. 7A and 7B set forth line drawings of an example optical elementand a fan of laser light projected from an ear probe having such anoptical element.

FIGS. 8A and 8B set forth line drawings of a further optical element anda resultant ring of laser light projected from an ear probe having suchan optical element.

FIG. 9 illustrates a skin target with partial lateral portions of ringsof laser light projected thereon.

FIG. 10 illustrates reflected laser light intensity varying in abell-curve shape with a thickness of a section of projected laser light.

FIG. 11 sets forth an image captured from reflections of laser lightreflected from a conical laser reflective optical element.

FIG. 12 sets forth a line drawing schematically illustratingtransforming ridge points to points in scanner space.

FIG. 13 sets forth a line drawing illustrating an examplethree-dimensional image of an ear canal constructed by use of a dataprocessor from a sequence of 2D images.

FIG. 14 sets forth a 3D image of a scanned ear created by use of anotoscanner and 3D imaging according to embodiments of the presentinvention.

FIG. 15 sets forth a line drawing of an otoscanner capable of detectingthe force with which the ear probe is pressed against a surface of thescanned ear for use in calculating a compliance value as an aid to amanufacturer in making comfortable and well fitting objects worn in theear.

FIG. 16 sets forth a further example otoscanner according to embodimentsof the present invention.

FIG. 17 sets forth a line drawing illustrating a method of determiningthe location and orientation in ear space of the ear drum of a scannedear according to a method of structure-from-motion.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Example otoscanning apparatus and methods according to embodiments ofthe present invention are described with reference to the accompanyingdrawings, beginning with FIG. 1A. FIG. 1A sets forth a line drawing ofan example otoscanner (100) having an otoscanner body (102). Theotoscanner body (102) includes a hand grip (104). The otoscanner body(102) has mounted upon it an ear probe (106), a tracking illuminationemitter (129 on FIG. 2), a plurality of tracking illumination sensors(108, not visible on FIG. 1A, visible on FIG. 2), and a display screen(110). The otoscanner body has mounted within it an image sensor (112).

The display screen (110) is coupled for data communications to the imagesensor (112), and the display screen (110) displays images of thescanned ear (126). FIG. 1A includes a callout (152) that schematicallyillustrates an example of the display screen (110) coupled for datacommunications to the image sensor (112) through a data communicationsbus (131), a communications adapter (167), a data processor (156), and avideo adapter (209). The displayed images can include video images ofthe ear captured by the image sensor (112) as the probe is moved withina scanned ear (126). The displayed images can include real-timeconstructions of 3D images of the scanned ear, such as the oneillustrated on FIG. 13. The displayed images can also include snapshotimages of portions of the scanned ear.

The display screen (110) is positioned on the otoscanner body (102) inrelation to the ear probe (106) so that when the ear probe (106) ispositioned for scanning, both the display screen (110) and the ear probe(106) are visible to any operator (103) of the otoscanner (100). In theexample of FIG. 1A, the display screen (110) positioned on theotoscanner body (102) in relation to the ear probe (106) so that whenthe ear probe (106) is positioned for scanning, both the display screen(110) and the ear probe (106) are visible to an operator operating theotoscanner (100) is implemented with the ear probe (106) mounted on thescanner body (102) between the hand grip (104) and the display screen(110) and the display screen (110) mounted on the opposite side of thescanner body (102) from the ear probe (106) and distally from the handgrip (104). In this way, when an operator takes the grip in theoperator's hand and position the probe to scan an ear, both the probeand the display are easily visible at all times to the operator.

In the example of FIG. 1A, the display screen (110) is positioned on theotoscanner body (102) in relation to the ear probe (106) so that whenthe ear probe (106) is positioned for scanning, both the display screen(110) and the ear probe (106) are visible to any operator (103) of theotoscanner (100). This is for explanation, and not for limitation. Infact, in some embodiments, the display screen (110) is not positioned onthe otoscanner body (102) in any particular relation to the ear probe(106). That is, in some such embodiments, during scanning the ear probeis not visible to the operator or the display screen is not visible tothe operator. The ear probe may therefore be located anywhere on theotoscanner body with respect to the display screen if both areintegrated into the otoscanner. And furthermore, in some embodiments,the otoscanner may not even have an integrated display screen.

FIG. 1A includes a callout (105) that illustrates the ear probe (106) inmore detail. The ear probe (106) includes a wide-angle lens (114) thatis optically coupled to the image sensor (112), with the lens and thesensor oriented so as to capture images of surfaces illuminated by lightfrom laser and non-laser light sources in the probe. In the exampleotoscanner probe (106) of FIG. 1A, the wide angle lens (114) has asufficient depth of field so that the entire portion of the surface ofan ear (126) illuminated by laser light is in focus at the image sensor(112). An image of a portion of the scanned ear is said to be in focusif light from object points on the surface of the ear is converged asmuch as reasonably possible at the image sensor (112), and out of focusif light is not well converged. The term “wide angle lens” as usedherein refers to any lens configured for a relatively wide field of viewthat will work in tortuous openings such as an auditory canal. Forexample, for an auditory canal, a 63 degree angle results in alens-focal surface offset about equal to the maximum diameter of theauditory canal that can be scanned with a centered ear probe. The focalsurface of a 60 degree lens (a fairly standard sized wide angle lens) isequal to the diameter, resulting in a forward focal surface of about 6mm, which typically is short enough to survive the second bend in anauditory canal which is at about a 6 mm diameter. For scanning auditorycanals, therefore, wide angle lenses typically are 60 degrees orgreater. Other functional increments include 90 degrees with its 2:1ratio allowing a forward focal surface distance of about 3 mm, allowingan ear probe to be fairly short. Lenses that are greater than 90 degreesare possible as are lenses that include complex optical elements withsideways only views and no forward field of view. According to someembodiments, laser light is emitted from the otoscanner probe in theform of a ring or in the form of a fan, and the wide angle lens providesthe same sufficient depth of field to portions of a scanned ear asilluminated by all such forms of laser.

The wide angle lens (114) can view relatively proximate lateral portionsof a surface with high precision due to overlap of its focal surfacewith a pattern of projected laser light. The term “focal surface” refersto a thickness within a range of focus of the wide angle lens that iscapable of achieving a certain base line resolution, such as being ableto discern a 50 micrometer feature or smaller. In an embodiment, forexample, lateral positioning of a pattern of projected laser lightwithin the focal surface can allow one pixel to be equivalent to about50 micrometers. Such a focal surface itself would have a bell curvedistribution of resolution that would allow variations in overlap orthickness of the focal surface and the width of the lateral portion ofreflected laser light which, as described in more detail below, has itsown curved distribution across its thickness.

Wide angle lenses (114) in embodiments typically have a reasonably lowdistortion threshold to meet resolution goals. Most wide angle lensescan be as high as −80 percent or −60 percent distortion that would needto be compensated by improved accuracy in other areas such as placementof the focal surface and lateral portion of projected patterns of laserlight. There is therefore no set threshold although collectively thevarious components are preferably tuned to allow a 50 micrometer orbetter resolution for lateral distances from the optical axis of thewide angle lens. A distortion of −40 percent or better provides aworkable field of view for scanning auditory canals.

The ear probe (106) includes a laser light source (116), a laser opticalelement (118), and a source of non-laser video illumination (120). Thelaser light source (116) delivers laser light (123) that illuminatessurfaces of a scanned ear (126) with laser light, and the videoillumination source delivers video illumination that illuminatessurfaces of a scanned ear with non-laser light (121). In the example ofFIG. 1A, the laser light source (116) in the ear probe is implemented asan optical fiber (130) that conducts laser light to the ear probe (106)from a laser outside the probe (106). In fact, in the example of FIG.1A, both sources of illumination (116, 120) are implemented with opticalfiber that conduct illumination from, for example, sources mountedelsewhere in the otoscanner body, a white light-emitting-diode (‘LED’)for the non-laser video illumination (121) and a laser diode or the likefor the laser light (123). For further explanation, an alternativestructure for the laser light source is illustrated in FIG. 6, where thelaser light source is implemented as an actual laser (158), such as, forexample, an on-chip laser diode, mounted directly on mounting structuresdisposed in the probe itself. In the example of FIG. 6, a laser powersource (160), electrical wiring, replaces the optical fiber (116 on FIG.1A) in the overall structure of the probe, connecting a power supplyoutside the probe to the laser (158). In the examples both of FIG. 1Aand FIG. 6, the laser light (123) is collimated by a laser opticalelement (118), and the non-laser video illumination (121) is diffused bya transparent top cap (127) mounted on the tip of the probe. Laserillumination from the laser light source (116) can be on continuouslywith the LED pulsed or both the laser and the LED can be pulsed, forexample.

The otoscanner (100) in the example of FIG. 1A provides a mode switch(133) for manual mode switching between laser-only mode, in which alaser-illuminated scan of an ear is performed without video, and avideo-only mode in which non-laser light is used to illuminate a scannedear and normal video of the ear is provided on the display screen (110).The laser light is too bright to leave on while capturing video images,however, so with manual switching, only one mode can be employed at atime. In some embodiments of the kind of otoscanner illustrated forexample in FIG. 1A, therefore, the image sensor is configured so as tocapture images at a video frame rate that is twice a standard videoframe rate. The frame rate is the frequency at which an imaging sensorproduces unique consecutive images called frames. Frame rate istypically expressed in frames per second. Examples of standard videoframe rates include 25 frames per second as used in the PhaseAlternating Line or ‘PAL’ video standard and 30 frames per second asused in the National Television System Committee or ‘NTSC’ videostandard. At twice a standard frame rate, video and laser-illuminatedimages can be captured on alternate frames while leaving the frame ratefor each set to a standard video rate. In such embodiments, thenon-laser video illumination (120, 121) is left on at all times, but thelaser light source (116) is strobed during capture by the image sensorof alternate video frames. Video frames are captured by the image sensor(112) when only the non-laser video illumination illuminates the scannedear, that is, on the alternate frames when the laser light source (116)is strobed off. Then laser-illuminated images for constructing 3D imagesare captured by the image sensor (112) only when strobed laser lightilluminates the scanned ear, that is, during the alternate frames whenthe laser light source (116) is strobed on, overwhelming the always-onnon-laser video illumination.

For further explanation, FIGS. 1B-1E set forth line drawings of furtherexample otoscanners, illustrating additional details of exampleembodiments. In the example of FIG. 1B, an otoscanner (100) includes abody (102), display (110), tracking sensors (108), and grip (104), allimplemented in a fashion similar to that of the otoscanner describes andillustrated above with reference to FIG. 1A. The example of FIG. 1Bincludes 5-inch radius arcs (157) that defines and connect the screentop to a grip bump profile on the back of the otoscanner body, thebottom of the grip to the bottom of the display screen, and the top of a45-degree cut at the bottom of the grip to the bottom of the displayscreen. In addition, the example of FIG. 1B includes a 20-inch radiusarc (161) that defines the overall curvature of the grip (104).

In the example of FIG. 1C, an otoscanner (100) includes a body (102),display (110), tracking sensors (108), and grip (104), all implementedin a fashion similar to that of the otoscanner describes and illustratedabove with reference to FIG. 1A. The example of FIG. 1C includes adescription of the grip (104) as elliptical in cross section, conformingto an ellipse (163) in this example with a major axis 1.25 inches inlength and a minor axis of 1.06 inches. The example of FIG. 1C alsoincludes a display screen 2.5 to 3.5 inches, for example, in diagonalmeasure and capable of displaying high-definition video. The displayscreen (110) is also configured with the capability of displaying imagesin portrait orientation until the otoscanner body is oriented forscanning an ear, at which time the display can change to a landscapeorientation. Indents (155) are provided around control switches (133)both on front and back of the grip (104) that guide operator fingers tothe control switches with no need for an operator takes eyes off thedisplay screen or the probe to look for the switches.

In the example of FIG. 1D, an otoscanner (100) includes a body (102),display (110), tracking sensors (108), and grip (104), all implementedin a fashion similar to that of the otoscanner describes and illustratedabove with reference to FIG. 1A. The example of FIG. 1D includes anillustration of the display screen (110) oriented at a right angle (165)to a central axis of the ear probe (106) so as to maintain the overallorientation of the display as it will be viewed by an operator.

In the example of FIG. 1E, an otoscanner (100) includes a body (102),tracking sensors (108), and grip (104), all implemented in a fashionsimilar to that of the otoscanner describes and illustrated above withreference to FIG. 1A. The example of FIG. 1E includes an illustration ofthe orientation of an array of tracking sensors (108) on the back of thedisplay, that is, on the opposite side of the otoscanner body from thedisplay screen, oriented so that the tracking sensor can sensereflections of tracking illumination from tracking targets fixed inposition with respect to a scanned ear. The tracking sensor are disposedbehind a window that is transparent to the tracking illumination,although it may render the tracking sensors themselves invisible innormal light, that is, not visible to a person. The example of FIG. 1Ealso includes a grip (104) whose length accommodates large hands,although the diameter of the grip is still comfortable for smallerhands. The example of FIG. 1E also includes a cable (159) that connectselectronic components in the otoscanner body (102) to components outsidethe body. The cable (159) balances the weight of the display block,which holds much of the weight of the otoscanner body. The use of thecable (159) as shown in FIG. 1E provides to an operator an overallbalanced feel of the otoscanner body.

Referring again to FIG. 1A, the image sensor (112) is also coupled fordata communications to a data processor (128), and the data processor(128) is configured so that it functions by constructing, in dependenceupon a sequence of images captured when the scanned ear is illuminatedby laser light and tracked positions of the ear probe inferred fromreflections of tracking illumination sensed by the tracking illuminationsensors, a 3D image of the interior of the scanned ear, such as, forexample the image illustrated in FIG. 13. For further explanation, FIG.2 sets forth a line drawing of an example otoscanner with a number oftracking illumination sensors (108) disposed upon the otoscanner body(102) so as to sense reflections (127) of tracking illumination (122)emitted from the tracking illumination emitter (129) and reflected fromtracking targets (124) installed at positions that are fixed relative tothe scanned ear (126). The tracking illumination sensors (127) arephotocells or the like disposed upon or within the opposite side of thedisplay block from the display and organized so as to distinguish anglesand brightness of tracking illumination reflected from tracking targets.In the example of FIG. 2, the tracking targets (124) are implemented asretroreflectors, and the tracking illumination (122) is provided from atracking illumination source or emitter (129), such as an LED or thelike, mounted on the otoscanner body (102). In at least someembodiments, the tracking illumination (122) is infrared.

In the example of FIG. 2, the tracking sensors (108) are mounteddirectly on or within the otoscanner (100). In other embodiments, thetracking sensors are mounted elsewhere, in other locations fixed withinscanner space, not on or within the otoscanner itself. In suchembodiments, a stand alone or separate tracking system can be used. Suchembodiments can include one or many tracking sensors, one or many lightsources. Some embodiments exclude tracking entirely, instead relying ofthe stability of an object to be scanned. To the extent that such anobject is an ear, then the person to whom the ear belongs must sit verystill during the scan. Other embodiments use a tripod for mounting thetracking systems of tracking illumination sensors.

The data processor (128) configured so that it constructs a 3D image ofthe interior of the scanned ear can be implemented, for example, by aconstruction module (169) of computer program instructions installed inrandom access memory (‘RAM’) (168) operatively coupled to the processorthrough a data communications bus. The computer program instructions,when executed by the processor, cause the processor to function so as toconstruct 3D images based on tracking information for the otoscope bodyor probe and corresponding images captured by the image sensor when asurface of a scanned ear is illuminated with laser light.

For explanation of a surface of a scanned ear illuminated with laserlight, FIG. 3A sets forth a line drawing of a projection onto a surfaceof an auditory canal of a ring of laser, the ring projected from aconical reflector (132 on FIG. 8A) into a plane which forms a brokenring (134) as the plane of laser light encounters the inner surface ofthe auditory canal. As the ear probe (106) moves through the auditorycanal (202), an image sensor in the otoscanner captures a sequence (135)of images of the interior of the auditory canal illuminated by rings ofprojected laser light. Each such image is associated with trackinginformation gathered by tracking apparatus as illustrated and describedwith regard to FIG. 2. A combination of such images and associatedtracking information is used according to embodiments of the presentinvention to construct 3D images of a scanned ear.

For further explanation of a surface of a scanned ear illuminated withlaser light, FIG. 3B sets forth a line drawing of a projection ontosurface of a pinna or aurical of a scanned ear of a fan (138) of laser,the fan projected from a diffractive laser lens (136 on FIG. 7A) into afan shape which illuminates the surface of the pinna, conforming to thesurface of the pinna as the fan of laser light encounters the pinna. Asan ear probe (106) is moved to scan the pinna, an image sensor in theotoscanner captures a sequence (137) of images of the surface of thepinna as illuminated by the fan (138) of projected laser light. Eachsuch image is associated with tracking information gathered by trackingapparatus as illustrated and described with regard to FIG. 2. Acombination of such images and associated tracking information is usedaccording to embodiments of the present invention to construct 3D imagesof a scanned ear.

For further explanation of construction of 3D images with an otoscanneraccording to embodiments of the present invention, FIG. 4 sets forth aflow chart illustrating an example method of constructing a 3D image ofa scanned ear. The method of FIG. 4 includes capturing (302), with animage sensor (112) of an otoscanner of the kind described above, asequence (304) of 3D images of surfaces of a scanned ear. The sequenceof images is a sequence of 2D images of surfaces of the scanned earilluminated with laser light as described above. The image sensorincludes an array of light-sensitive pixels, and each image (304) is aset of pixel identifiers such as pixel numbers or pixel coordinates witha brightness value for each pixel. The sequence of 2D images is used asdescribed to construct a 3D image.

The method of FIG. 4 also includes detecting (306) ridge points (308)for each 2D image. Ridge points for a 2D image make up a set ofbrightest pixels for the 2D image, a set that is assembled by scanningthe pixel brightness values for each 2D image and selecting as ridgepoints only the brightest pixels. An example of a 2D image is set forthin FIG. 10, illustrating a set of brightest pixels or ridge points (176)that in turn depicts a c-shaped broken ring of laser light reflectingfrom a surface of an auditory canal of a scanned ear.

The method of FIG. 4 also includes transforming (318) the ridge pointsto points in scanner space. The transforming (318) in this example iscarried out by use of a table of predefined associations (312) betweeneach pixel in the image sensor (112) and corresponding points in scannerspace. Each record of table (312) represents an association between apixel (326) of the image sensor (112) and a point in scanner space (200on FIG. 2). In the example of table (312), n pixels are identified withnumbers, 1, 2, 3, . . . , n−1, n. The pixels of the image sensor can beidentified by their x,y coordinates in the image sensor itself, or inother ways as will occur to those of skill in the art. Thecorrespondence between pixels and points in scanner space can beestablished as described and illustrated below with reference to FIG.12, triangulation according to equations 2-8. Such triangulation can becarried out by data processor and algorithm for each pixel of eachcaptured frame from the image sensor, although that is computationallyburdensome, it is feasible with a fast processor. As a lesscomputationally intense alternative, the triangulation can be carriedout once during manufacture or calibration of an otoscanner according toembodiments of the present invention, with the results stored, forexample, in a structure similar to Association table (312). Using suchstored associations between pixels and points in scanner space, theprocess of transforming (310) ridge points to points in scanner space iscarried out with table lookups and the like rather than real timetriangulations.

The example table (312) includes two columns, one labeled ‘Pixel’ thatincludes values identifying pixels, and another labeled ‘Coordinates’that identifies the locations in scanner space that correspond to eachpixel. Readers will recognize that in embodiments in which the recordsin table (312) are sorted as here according to pixel location, then the‘Pixel’ column actually would not be needed because the position ofcoordinates in the ‘Coordinates’ columns would automatically index andidentify corresponding pixels. In embodiments that omit the ‘Pixel’columns based on such reasoning, the Associations table (312) iseffectively simplified to an array of coordinates. In fact, the datastructures of table and array are not limitation of the presentinvention, but instead are only examples of data structures by which canbe represented correspondence between pixels and points in scannerspace. Readers will recognize that many data structures can be so used,including, for example, C-style structures, multi-dimensional arrays,linked lists, and so on.

The method of FIG. 4 also includes transforming (318) the points (314)in scanner space (200 on FIG. 2) to points (320) in ear space (198 onFIG. 2). This transforming (318) is carried out according to arelationship between an origin (151 on FIG. 2) of a coordinate systemdefining scanner space (200 on FIG. 2) and an origin (150 on FIG. 2) ofanother coordinate system defining ear space (198 on FIG. 2). That is,scanner space is both translated and rotated with respect to ear space,and this relationship differs from frame to frame as an otoscanner ismoved in ear space during a scan. The relationship for each frame isexpressed as Tensor 1.

$\begin{matrix}\begin{bmatrix}R_{11} & R_{12} & R_{13} & T_{1} \\R_{21} & R_{22} & R_{23} & T_{2} \\R_{31} & R_{32} & R_{33} & T_{3} \\0 & 0 & 0 & 1\end{bmatrix} & {{Tensor}\mspace{14mu} 1}\end{matrix}$

The T values in Tensor 1 express the translation of scanner space withrespect to ear space, and the R value express the rotation of scannerspace with respect to ear space. With these values in Tensor 1, thetransformation of points in scanner space to points in ear space iscarried out according to Equation 1.

$\begin{matrix}{\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime} \\1\end{bmatrix} \equiv {\begin{bmatrix}R_{11} & R_{12} & R_{13} & T_{1} \\R_{21} & R_{22} & R_{23} & T_{2} \\R_{31} & R_{32} & R_{33} & T_{3} \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}x \\y \\z \\1\end{bmatrix}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

Equation 1 transforms by matrix multiplication with Tensor 1 a vectorrepresenting point x,y,z in scanner space into a vector representingpoint x′,y′,z′ in ear space. The transforming (318) of points in scannerspace to points in ear space can be carried out by establishing Tensor 1for each image scanned from the image sensor and applying Equation 1 toeach point (314) in scanner space represented by each pixel in eachimage.

The method of FIG. 4 also includes summing (321) the points in ear spaceinto a 3D image (325) of an ear. The results of such summing are shownschematically in FIG. 13, and an actual 3D image of a scanned ear is setforth in FIG. 14. The image in FIG. 14 was created using the transformedpoints in ear space as such to display a 3D image. Such a set of pointsis a mathematical construct. In 3D computer graphics generally, 3Dmodeling is developing a mathematical representation of athree-dimensional surface of an object (living or inanimate). Theproducts of such processes are called 3D images or 3D models. Suchimages can be displayed as a two-dimensional image through a processcalled 3D rendering or used in a computer simulation of physicalphenomena. Such an image or model can also be used to create an actualthree-dimensional object of a scanned object, such as a scanned ear,using a 3D model as an input to a CAD/CAM process or a 3D printingdevice.

The method of FIG. 4 also includes determining (324) whether a scan iscomplete. This determination is carried out by comparing the summed setof points in ear space that now make up a 3D image of the scanned earfor completeness by comparing the 3D image with scanning requirements(322) as specified for a particular, pre-selected class, make, and modelof an object to be worn in the ear, an auditory bud, in-ear headphone,hearing aid, or the like. If the scan is incomplete, portions of the 3Dimage will not meet the scanning requirements as specified for theclass, make, and model of the object to be worn in the ear. Often theincomplete portions of the 3D image will appear as holes in the 3Dimage.

Not all objects worn in the ear require the same portions of the ear tobe scanned. Scanning requirements (322) as specified for a particular,pre-selected class, make, and model of an object to be worn in the ear.For example, behind-the-ear hearing aids use a mold that requires theconcha of the ear to be scanned, in-the-ear hearing aids require more ofthe auditory canal to be scanned, invisible-in-the-ear hearing aidsrequire even more of the auditory canal to be scanned than thein-the-ear hearing aids. Each of these different classes of hearing aids(behind-the-ear, in-the-ear, and invisible-in-the-ear) may be used todetermine whether a scan is complete by also determining which portionsof the ear are to be scanned for the particular class of the hearingaid. Within each class of hearing aid or other object to be worn in theear the make and model may also affect which portions of the ear are tobe scanned to make up a complete scan of the ear. Each of thesedifferent makes and models within a class of hearing aids may also beused to determine whether a scan is complete including determining whichportions of the ear are to be scanned for the particular class of thehearing aid.

For further explanation, FIG. 5 sets forth a line drawing illustratingadditional example features of an ear probe (106) and image sensor (112)of an otoscanner according to embodiments of the present invention. Theprobe (106) of FIG. 5 has a wide angle lens (114) that includes a numberof lens elements (115) and spacers (125). The wide angle lens (114) ofFIG. 5 has a sufficient depth of field so that the entire portion of theinterior surface of the ear (126) illuminated by laser light is in focusat the image sensor (112). An image of a portion of the ear is said tobe in focus if light from object points on the interior of the ear isconverged as much as reasonably possible at the image sensor, and out offocus if light is not well converged. Supporting the wide angle lens 114of FIG. 5 is a focusing screw 164 that when turned adjusts the focus ofthe wide angle lens 114 for improved accuracy and for compensating formanufacturing tolerances.

The probe (106) of FIG. 5 also includes a laser light source (116) and alaser optical element (118). In the example of FIG. 5 the laser lightsource (116) is a fiber optic cable carrying laser light from a laserwithin the body of the otoscanner to the laser optical element. Asmentioned above, in some embodiments of otoscanners according to thepresent invention, the laser optical element (118) may include a conicallaser reflective optical element. In such embodiments, the lens elements(115) of the wide angle lens (114) of FIG. 5 has sufficient depth offield so that the portion of the interior surface of the ear (126)illuminated by laser light is in focus at the image sensor (112) whenthe interior surface of the ear is illuminated by a ring of laser lightcreated by use of the conical laser reflective optical element andprojected through the transparent side walls of the window (166). Insome other embodiments of the present invention, the laser opticalelement (118) may include a diffractive laser optic lens. In suchembodiments, the lens elements (115) of the wide angle lens (114) ofFIG. 5 has sufficient depth of field so that the portion of the interiorsurface of the ear (126) illuminated by laser light is in focus at theimage sensor (112) when the interior surface of the ear is illuminatedby a fan of laser light created by use of a diffractive laser optic lensand projected through the front of the transparent window (116).

In the example of FIG. 5, the image sensor (112) operates at a videoframe rate that is twice a standard video frame rate. By operating attwice a standard video frame rage the image sensor may capture usablevideo of the scanned ear as well as capture images of the scanned earfor constructing 3D images of the scanned ear. In the example of FIG. 5,therefore, the laser light source (116) is strobed during capture by theimage sensor (112) of alternate video frames thereby allowing everyother video image to be a 2D image for constructing 3D images. The 2Dimages for constructing 3D images are captured by the image sensor onlywhen the strobed laser light illuminates the scanned ear. Video framesare captured by the image sensor (112) when only the non-laser videoillumination from the video illumination source (120) illuminates thescanned ear.

In the example of FIG. 5, the laser light source (116) of FIG. 5completely overpowers the video illumination source (120). The videoillumination source (12) therefore 4 may remain on such that non-laservideo illumination is on during operation of the otoscanner. Therefore,when the laser light source (116) is strobed, it completely overpowersthe video illumination and each time the laser light source illuminatesthe scanner ear with laser light images captured by the image sensor are2D images of the scanned ear for construction of a 3D image.

For further explanation, FIG. 6 sets forth a line drawing of an exampleear probe (106) of an otoscanner according to embodiments of the presentinvention. The ear probe (106) of FIG. 6 is similar to the ear probe ofFIG. 1A in that it includes a lens (114) with lens elements (115) andspacers (125), a lens tube (117) a video illumination source, a probewall (119), and a laser optical element (118). The field of view of theillustrated embodiment, shown by dotted lines, is approximately 150degrees, although the light pattern (123) may extend laterally out atright angles to the optical axis of the wide angle lens (114). Angles upto 180 degrees are possible but wider angles can be increasinglydifficult to minimize distortion. The ear probe (106) of FIG. 6 differsfrom the ear probe of FIG. 1A in that the laser light source of the earprobe of FIG. 6 is a laser (158) mounted in the probe (106) itself. Inthe example of FIG. 6 the laser (158) is mounted in the probe and powerto the laser is proved by a laser power source (160) delivering powerfrom within the otoscanner body. In some embodiments, the laser may be amounted on a bare die allowing the laser to be placed directly on aprinted circuit board in the ear probe.

As mentioned above, otoscanners according to embodiments of the presentinvention may be configured to project a ring of laser light radiallyfrom the tip of the distal end of the ear probe, project a fan of laserlight forward from the tip of the distal end of the ear probe, orconfigured to project other shapes of laser light as will occur to thoseof skill in the art. For further explanation, therefore, FIGS. 7A and 7Bset forth line drawings of an optical element (118) useful in scannersaccording to embodiments of the present invention and a resultant fan oflaser light (138) projected from an ear probe having such an opticalelement. The laser optical element (118) of FIG. 7A comprises adiffractive laser optic lens (136). In the example of FIG. 7A, the laserlight source (116) and the diffractive laser optic lens (136) areconfigured so that when illuminated by the laser light source (116) thediffractive laser optic lens (136) projects upon an interior surface ofthe ear a fan (138) of laser light at a predetermined angle (140) withrespect to a front surface (142) of the diffractive laser optic lens(136). In the example of FIGS. 7A and 7B, laser light from the source oflaser light (116) is focused by a ball lens (170) on the diffractivelaser optic lens (136). The diffractive laser optic lens (136) diffractsthe laser light into a fan (138) of laser light. The diffractive laseroptic lens (136) is manufactured to diffract the laser light at apredetermined angle (140) from its front surface (142) into a fan oflaser light (138) as illustrated in FIGS. 7A and 7B.

As mentioned above, otoscanners according to embodiments of the presentinvention may be configured to project a ring of laser light radiallyfrom the tip of the distal end of the ear probe. For furtherexplanation, therefore, FIGS. 8A and 8B set forth line drawings of anoptical element (118) useful in scanners according to embodiments of thepresent invention and a resultant ring of laser light (134) projectedfrom an ear probe having such an optical element. The laser opticalelement (118) of FIG. 8A includes a conical laser-reflective opticalelement (132). In the example of FIG. 8A the laser light source (116)and the conical laser-reflecting optical element (132) are configured sothat the conical laser-reflecting optical element (132), whenilluminated by the laser light source (116), projects a broken ring(134) of laser light upon an interior surface of the ear when the earprobe is positioned in the ear. In the example of FIGS. 6A and 6B, laserlight from the laser light source (116) is focused by a ball lens (170)onto the conical laser reflective optical element (132). The conicallaser reflective optical element (132) reflects the laser light into aring of laser light (134) as illustrated in FIGS. 8A and 8B.

In the examples of FIGS. 8A and 8B the ring of laser light is brokenbecause the conical laser reflective optical element (132) is mounted ina fashion that blocks a portion of the laser light reflected by theoptical element. In alternate embodiments, however, the ring of laserlight reflected by the conical laser reflective optical element (132) isunbroken as will occur to those of skill in the art.

Referring to FIG. 9, a skin target is shown with partial lateralportions 20 of rings of laser light projected thereon for the purpose ofdetermining how the laser light will project upon skin and its locationbe marked. A perpendicular section of one of the lateral portions, asshown in FIG. 10, illustrates the fact that the reflected laser lightintensity (y-axis) varies in a bell-curve shape with the thickness(x-axis) of the section. Thus, the partial lateral portion 20 mayinclude an edge 22 of the light pattern as well as a ridge 24 of thelight pattern. These landmarks may be used to determine the position ofthe lateral portion 20 in a coordinate system defining an ear space. Forexample, one of the aforementioned landmarks could be found (such as bya ridge detecting function of a data processor) or an inside edge of thelateral portion or an outside edge of the lateral portion. Or, anaverage of the inside and outside portions may be used.

For further explanation, FIG. 11 sets forth an image captured fromreflections of laser light reflected from a conical laser reflectiveoptical element (132) radially from the tip of the ear probe of anotoscanner according to embodiments of the present invention. Thecaptured image of FIG. 11 forms a c-shaped broken ring of pixels ofhighest intensity. Along the outside and inside of the broken ring (180)are pixels of intensity defining an edge as mentioned above. In betweenthe edges (178) of the broken ring are pixels of higher intensity thatdefine a ridge. The ridge (176) is a collection of ridge points thatcomprise a set of brightest pixels for the captured 2D image.

Constructing a 3D image of the interior of a scanned ear according toembodiments of the present invention for a sequence of 2D images of theear such as the image of FIG. 11 includes detecting ridge points foreach 2D image. Detecting ridge points in the example of FIG. 11 includesidentifying a set of brightest pixels for the 2D image. In the exampleof FIG. 11, ridge points are detected as a set of brightest pixels alongthe ridge (176) of the image (180). Detecting ridge points may becarried out by scanning across all pixels in a row on the image sensorand identifying a pixel whose intensity value is greater than theintensity values of pixels on each side. Alternatively, detecting aridge point may be carried out by identifying range of pixels whoseaverage intensity values are greater than the intensity values of arange of pixels on each side and then selecting one of the pixels in therange of pixels with greater average intensity values. As a furtheralternative, detecting ridge points can be carried out by taking thebrightest pixels from a purposely blurred representation of an image, atechnique in which the pixels so selected generally may not be theabsolute brightest. An even further alternative way of detecting ridgepoints is to bisect the full-width half maximum span of a ridge atnumerous cross sections along the ridge. Readers will recognize fromthis description that constructing a 3D image in this example is carriedout with some kind of ridge detection. In addition to ridge detection,however, such construction can also be carried out using edge detection,circle detection, shape detection, snakes detection, deconstructiontechniques, and in other ways as may occur to those of skill in the art.

Constructing a 3D image of the interior of a scanned ear according toembodiments of the present invention for a sequence of 2D images alsoincludes transforming, in dependence upon a predefined associationbetween each pixel in the image sensor and corresponding points inscanner space, the ridge points to points in scanner space as describedwith reference to FIG. 11 and transforming, in dependence upon arelationship between an origin of a coordinate system defining scannerspace and an origin of another coordinate system defining ear space, thepoints in scanner space to points in ear space as described withreference to FIG. 13.

For further explanation, FIG. 12 sets forth a line drawing schematicallyillustrating transforming, in dependence upon a predefined associationbetween each pixel in the image sensor and corresponding points inscanner space, the ridge points to points in scanner space. FIG. 12schematically shows an embodiment for calculation of the radial distanceof the lateral portion from the optical axis of the probe as implementedby a data processor. The position can be determined by triangulation, asshown in equations 2-8.

$\begin{matrix}{\frac{h}{S^{\prime}} \equiv \frac{R}{S}} & {{Equation}\mspace{14mu} 2} \\{R = \frac{hS}{S}} & {{Equation}\mspace{14mu} 3} \\{\frac{S^{\prime}}{S} = M} & {{Equation}\mspace{14mu} 4} \\{R = \frac{h}{M}} & {{Equation}\mspace{14mu} 5} \\{{\Delta \; R} = \frac{\Delta \; h}{M}} & {{Equation}\mspace{14mu} 6} \\{\theta_{m\; i\; n} = {{Tan}^{- 1}\left( \frac{R_{m\; i\; n}}{S} \right)}} & {{Equation}\mspace{14mu} 7} \\{\theta_{{ma}\; x} = {{Tan}^{- 1}\left( \frac{R_{{ma}\; x}}{S} \right)}} & {{Equation}\mspace{14mu} 8}\end{matrix}$

In the example of FIG. 12 and in equations 2-8, scanner space isoriented so that its Z axis is centered and fixed as the central axis ofan ear probe, looking end-on into the probe, here also referred to asthe imaging axis. In this example, therefore, the ratio of the distanceR from the imaging axis of a laser-illuminated point to the distance Sbetween the laser plane and the lens is equal to that of the distance hfrom the center of the image sensor to the distance S′ between the imagesensor surface and the lens. Magnification M is the ratio of S′ and S.When the distances S and S′ between the lens and laser plane, and lensto image sensor are known, equations 2-8 can reconstruct the geometry ofilluminated points in scanner space. These equations also denote thatfor a focal surface such as a plane, there is a 1:1 mapping of points inscanner space to pixel locations on the image sensor.

The image sensor 112 may be implemented in complementary-symmetrymetallic-oxide-semiconductor (‘CMOS’) sensor, as a charge-coupled device(CCD), or with other sensing technology as may occur to those of skillin the art. A CMOS sensor can be operated in a snapshot readout mode orwith a rolling shutter when the scan along the Z-axis is incremented orstepped synchronously to effect a readout of a complete frame. Similarincrementing or stepping may be used for a CCD operated with interlacingscans of image frames.

Constructing a 3D image of the interior of a scanned ear according toembodiments of the present invention also often includes transforming,in dependence upon a relationship between an origin of a coordinatesystem defining scanner space and an origin of another coordinate systemdefining ear space, the points in scanner space to points in ear space.For further explanation, therefore, FIG. 13 sets forth a line drawingillustrating an exemplary three-dimensional image (182) of an ear canalconstructed from a sequence of 2D images by a data processor. In theexample of FIG. 13, each of the 2D images (186) includes a set oftransformed ridge points. The transformed ridge points are the result oftransforming, in dependence upon a relationship between an origin of acoordinate system defining scanner space and an origin of anothercoordinate system defining ear space, the points in scanner space topoints in ear space as described with reference to FIG. 13.Transforming, in dependence upon a relationship between an origin of acoordinate system defining scanner space and an origin of anothercoordinate system defining ear space, the points in scanner space topoints in ear space may be carried out by as described and illustratedabove with reference to FIG. 4.

For further explanation, FIG. 14 sets forth a 3D image of a scanned earcreated by use of an otoscanner and 3D imaging according to embodimentsof the present invention. The 3D image of FIG. 14 includes a 3Ddepiction of the concha (192), the aperture (188) of the ear, the firstbend (190) of the ear canal, the second bend of the ear canal and thelocation of the ear drum (196). The 3D image of FIG. 14 may be used by amanufacturer to provide custom fit hearing aids, custom fit ear buds forpersonal listening devices, custom fit headphones, and other objectscustom fit to the scanned ear and worn in the ear.

The density of portions of the skin making up the ear varies from personto person. The density of portions of the skin making up the ear alsovaries across the portions of the ear. That is, some people have earswith skin that is more compliant in certain areas of the ear thanothers. The compliance of the skin of an ear is a factor in determiningwhether a custom hearing aid, mold, or other object worn in the ear iscomfortable to its wearer while still providing a proper fit within theear. Compliance information may be provided to a manufacturer for usemaking a comfortable and well fitting hearing aid, mold, or other objectworn in the ear. For further explanation, therefore, FIG. 15 sets fortha line drawing of an otoscanner capable of detecting the force withwhich the ear probe is pressed against a surface of the scanned ear foruse in calculating a compliance value as an aid to a manufacturer inmaking comfortable and well fitting objects worn in the ear. Theotoscanner (100) of FIG. 15 is similar to the otoscanner of FIGS. 1 and2 in that the otoscanner has a body (102), an ear probe (106), videoillumination source (120) carrying video illumination from a non-laserlight emitter (220), a laser light source for a conical reflectiveoptical element (116 a) carrying laser light from a laser (158 a) in thebody (102) of the otoscanner (100), a laser light source for adiffractive optical lens (116 b) carrying light from a laser (158 b) inthe body (102) of the otoscanner (100) and so on.

The otoscanner (100) of FIG. 15 differs from the otoscanner of FIGS. 1and 2 in that the otoscanner body (102) has mounted within it pressuresensors (144) operably coupled to the ear probe (106). In the example ofFIG. 15, the pressure sensors (144) are coupled for data communicationsto the data processor (128) and pressure sensors detect the force withwhich the ear probe (106) is pressed against a surface of the scannedear. In some embodiments, the probe is implemented as entirely rigidwhen scanning. In other embodiments, the probe is implemented assomewhat moveable against pressure sensors for compliance measurements.And some embodiments implement a probe that is alternately both rigidand moveable, providing a locking mechanism that maintains the probe asrigid for optical scanning and allows the probe to move against apressure sensor when unlocked for ascertaining a compliance value.

The otoscanner (100) is also configured to track positions of the earprobe inferred from reflections of tracking illumination sensed by thetracking illumination sensors (108). The tracked positions are used toidentifying the displacement through which the ear probe (106) moveswhen pressed against the surface of the scanned ear. The data processor(128) of FIG. 15 is further configured so that it functions bycalculating a compliance value in dependence upon the detected force andthe tracked displacement. The compliance value may be implemented as asingle value or range of values dependent upon the detected force andthe identified displacement when the probe is pressed against thesurface of the scanned ear.

To facilitate the detection of the force when the probe is pressedagainst the surface of the scanned ear, the otoscanner body (102) hasmounted within it pressure sensors (144) operably coupled to the earprobe (106). The tracking sensors (108), the image sensor (112), theprobe (106) and lens of the otoscanner (100) of FIG. 15 are all mountedon a rigid chassis (146) that is configured to float within theotoscanner body (102). The pressure sensors (144) are mounted within theotoscanner (100) between the rigid chassis (146) and the otoscanner body(102). The rigid chassis (146) is floated in the body (102) of theotoscanner (100) in that the rigid chassis (146) may move relative tothe body (102) of the otoscanner (100) when the probe (106) is pressedagainst the surface of the ear.

In example otoscanners described above, the functionality of theotoscanner is described as residing within the body of the otoscanner.In some embodiments of the present invention, an otoscanner may beconfigured with a wireline connection to a data processor (128) in acomputer (202) available to an operator of the otoscanner. For furtherexplanation, therefore, FIG. 16 sets forth a further example otoscanneraccording to embodiments of the present invention that includes anotoscanner body (102) with a wireline connection (148) to a dataprocessor (128) implemented in a computer (204). In the example of FIG.16 the elements of the otoscanner are distributed between the otoscannerbody (102) and the computer (204). In the example of FIG. 16, thetracking targets (124) are fixed to a headband worn by the person whoseear (126) is being scanned.

The data processor (128) in the computer (204) of FIG. 16 includes atleast one computer processor (156) or ‘CPU’ as well as random accessmemory (168) (‘RAM’) which is connected through a high speed memory busand bus adapter to processor the (156) and to other components of thedata processor (128). The data processor (128) of FIG. 16 also includesa communications adapter (167) for data communications with othercomputers and with the otoscanner body (102) and for data communicationswith a data communications network. Such data communications may becarried out serially through RS-232 connections, through external busessuch as a Universal Serial Bus (‘USB’), through data communicationsnetworks such as IP data communications networks, and in other ways aswill occur to those of skill in the art. Communications adaptersimplement the hardware level of data communications through which onecomputer sends data communications to another computer, directly orthrough a data communications network. The example data processor FIG.16 includes a video adapter (209), which is an example of an I/O adapterspecially designed for graphic output to a display device (202) such asa display screen or computer monitor.

In the example of FIG. 16, the image sensor (112) is illustrated incallout (156) as residing within the otoscanner body as well as beingillustrated in callout (128) as residing in the data processor. An imagesensor useful in embodiments of the present invention illustrated inFIG. 16 may reside in either location, or as illustrated in callout(156) or in the computer (202) itself.

In the example of FIG. 16, a display screen (202) on the computer (204)may display images of the scanned ear scanned ear illuminated only bynon-laser video illumination (120). The display screen (202) on thecomputer (113) may also display 3D images of the scanned ear constructedin dependence upon a sequence of images captured by the image sensor asthe probe is moved in the scanned ear. In such examples images capturedby an image sensor (112)

Stored in RAM (168) in the data processor (128) of FIG. 16 is aconstruction module, a module of computer program instructions forconstructing 3D images of the scanned ear in dependence upon a sequenceof images captured by the image sensor (112) as the probe is moved inthe scanned ear. The construction module (169) is further configured todetermine the position of the probe (106) in ear space when the probe ispositioned at the aperture of the auditory canal of the scanned ear(126) and setting the position of the probe at the aperture of theauditory canal of the scanned ear as the origin of the coordinate systemdefining ear space.

Not all hearing aids, molds, or other objects worn in the ear requirethe same portions of the ear to be scanned. That is, some objects wornin the ear are small, some are large, some are placed deeper in the earthan others, and so on. As such, stored in RAM (168) in the dataprocessor of FIG. 16 is a completion module (206) a module of computerprogram instructions for determining whether a scan is complete independence upon a class, make, and model, of a hearing aid or otherobjects worn in the ear. The completion module (208) has a database ofclasses, makes, and models of hearing aids or other objects worn in theear. The classes, makes, and models identify the proper portions of theear to be scanned. The completion module is configured to identify fromthe 3D image of the ear constructed by the construction module (169)whether the 3D image includes scanned portions of the ear required forthe manufacture of a particular class, make and model of a hearing aidor other object worn in the ear. The completion module (208) is alsoconfigured to determine whether portions of the ear have simply not beenscanned at all. Such portions may appear as holes in the 3D image of theear.

There is a danger to an ear being scanned if a probe or other object isinserted too deeply in the ear. For example, an ear drum may be damagedif it comes into contact with a probe. Also stored in RAM (206),therefore, is a safety module (206), a module of computer programinstructions for safety of use of the otoscanner (100) of FIG. 16. Thesafety module (206) of FIG. 16 has a database of previously recordedstatistics describing typical ear sizes according to human demographicssuch as height weight, age and other statistics of the humans. Thesafety module (206) also has currently recorded demographic informationregarding a person whose ear is being scanned. The safety module infers,from a tracked position of the ear probe (106), previously recordedstatistics describing typical ear sizes according to human demographics,and currently recorded demographic information regarding a person whoseear is scanned, the actual present position of the ear probe in relationto at least one part of the scanned ear. The safety module is configuredto provide a warning when the probe moves within a predefined distancefrom the part of the scanned ear. Such a warning may be implemented as asound emitted from the otoscanner (100), a warning icon on a displayscreen of the otoscanner (100) or computer, or any other warning thatwill occur to those of skill in the art.

Those of skill in the art will recognize that the ear is flexible andthe shape of the ear changes when the mouth of the person being scannedis open and when it is closed. To facilitate manufacturing a hearingaid, mold or other object worn in the ear in the example of FIG. 16, anoperator scans the ear with the otoscanner of FIG. 16 with the mouthopen and then with the mouth closed. 3D images of the ear constructedwhen the mouth is open and also when the mouth is closed may then beused to manufacture a hearing aid, mold, or other object worn in the earthat is comfortable to the wearer when the wearer's mouth is open andwhen it is closed. The construction module (169) of the data processor(128) of FIG. 16 is therefore configured to construct the 3D image ofthe scanned ear by constructing the 3D image in dependence upon asequence of images captured by the image sensor as the probe is moved inthe scanned ear with mouth open. The construction module (169) of thedata processor (128) of FIG. 16 is also configured to construct the 3Dimage of the scanned ear by constructing the 3D image in dependence upona sequence of images captured by the image sensor as the probe is movedin the scanned ear with mouth closed.

The ear drum of a scanned ear is not always in the same place ororiented in the same way relative to an ear. That is, the location andorientation of ear drums differ for different people. Otoscannersaccording to embodiments of the present invention therefore may beconfigured to construct a 3D image of the interior of the scanned earthat includes determining the location and orientation in ear space ofthe ear drum of the scanned ear. For further explanation, FIG. 17 setsforth a line drawing illustrating a method of determining the locationand orientation in ear space of the ear drum of a scanned ear accordingto a method of structure-from-motion. In the example of FIG. 17, theforward field of view (FFOV) as captured through the probe by theotoscanner will see the ear drum in multiple video frames as the probeis moved through the ear. Because the otoscanner tracks the position andorientation of the probe relative to a coordinate system on the head(‘ear space’), the otoscanner's data processor can usestructure-from-motion to reconstruct the location and direction of theear drum. Consider one point on the ear drum X that is readilyidentifiable, such as the umbo, the most depressed part of the concavesurface of the ear drum. Referring to the illustration of FIG. 17,consider a point X on the ear drum that is seen on two images N and N+1.On Image N, point X is seen at a pixel location X_(L), and X falls on aray O_(L) X_(L). From a single image, however, it is not known how faraway X is: it could be at X1, X2, X3, etc. Having a second image (ImageN+1) allows computation of the distance. In Image N+1, the point on theear drum X is seen at pixel location X_(R). It follows that X is on theray O_(R)-X_(R). The otoscanner's data processor uses trackinginformation to transform the direction of these two rays into directionsin ear space. Computing the intersection of the rays in ear space yieldsthe location of the point on the ear drum X.

Readers will recognize that the particular structure-from-motiontechnique just described is not the only way of determining the locationand orientation in ear space of the ear drum of the scanned ear. Inembodiments, for example, a laser beam provided outside the lenses in aprobe is directed parallel to the central axis of the wide angle lens,and the laser produces a dot on the ear drum. In such embodiments, thelocation of the dot in scope space is determined withstructure-from-motion from a single image. In such embodiments, the dotmay be somewhat out of focus because the ear drum can be outside of thevolume of good focus for the wide angle lens.

As mentioned above, example embodiments have been described in terms ofotoscanners with particular reference to scanning ears. This is forexplanation and not for limitation. In fact, readers will recognize thatscanners according to embodiments of the present invention may be usedto scan almost any 3D surface of live or inanimate objects both internaland external.

Exemplary embodiments of the present invention are described largely inthe context of a fully functional otoscanner and system for scanning anear. Readers will recognize, however, that aspects of the presentinvention also may be embodied in a computer program product disposedupon computer readable storage media for use with any suitable dataprocessing system. Such computer readable storage media may be anystorage medium for machine-readable information, including magneticmedia, optical media, or other suitable media. Examples of such mediainclude magnetic disks in hard drives or diskettes, compact disks foroptical drives, magnetic tape, and others as will occur to those ofskill in the art. Persons skilled in the art will immediately recognizethat any computer system having suitable programming means will becapable of executing aspects of the invention. Persons skilled in theart will recognize also that, although some of the exemplary embodimentsdescribed in this specification are oriented to software installed andexecuting on computer hardware, nevertheless, alternative embodimentsimplemented as firmware or as hardware are well within the scope of thepresent invention.

It will be understood from the foregoing description that modificationsand changes may be made in various embodiments of the present inventionwithout departing from its true spirit. The descriptions in thisspecification are for purposes of illustration only and are not to beconstrued in a limiting sense. The scope of the present invention islimited only by the language of the following claims.

What is claimed is:
 1. A tracker for scanner motion, the trackercomprising: a tracking illumination emitter mounted on a scanner, thescanner including imaging apparatus that captures a sequence of imagesof a scanned object as the scanner moves with respect to the scannedobject; tracking targets installed off the scanner at positions that arefixed relative to a scanned object; one or more tracking illuminationsensors mounted on the scanner, the tracking illumination sensorssensing reflections of tracking illumination emitted from the trackingillumination emitter and reflected from the tracking targets as thescanner moves in the process of scanning the scanned object; and one ormore data processors, at least one of the data processors coupled fordata communications to the tracking illumination sensors, the dataprocessor determining, as the scanner moves, tracked positions of thescanner based upon values of the reflections.
 2. The tracker of claim 1further comprising at least one of the processors configured so that it:determines the position of the scanner in object space when the scanneris positioned at a predetermined position with respect to the scannedobject; and sets the position of the scanner at the predeterminedposition as the origin of a coordinate system defining object space. 3.The tracker of claim 1 wherein the scanned object is an ear, and thetracker further comprises at least one of the processors configured sothat it: determines the position of the scanner in object space when theprobe is positioned at the aperture of the auditory canal of the scannedear; and sets the position of the scanner at the aperture of theauditory canal of the scanned ear as the origin of a coordinate systemdefining object space.
 4. The tracker of claim 1 wherein the trackingtargets comprise retroreflectors.
 5. The tracker of claim 1 wherein thetracking illumination is infrared light.
 6. The tracker of claim 1further comprising at least one of the data processors configured sothat it constructs a 3D image of the scanned object based upon thesequence of images and the tracked positions of the scanner.
 7. Thetracker of claim 1 further comprising at least one of the dataprocessors configured so that it constructs a 3D image based upon thesequence of images and the tracked positions of the scanner, with thesequence of images implemented as a sequence of 2D images, including,for the sequence of 2D images from the image sensor of the object takenwhen the object is illuminated by imaging light from the scanner:detecting ridge points for each 2D image, including identifying a set ofbrightest pixels for each 2D image, each set depicting a c-shaped brokenring of imaging light reflecting from a surface of the scanned object;transforming, in dependence upon a predefined association between eachpixel in the image sensor and corresponding points in scanner space, theridge points to points in scanner space; and transforming, in dependenceupon a relationship between an origin of a coordinate system definingscanner space and an origin of another coordinate system defining objectspace, the points in scanner space to points in object space.
 8. Thetracker of claim 1 further comprising at least one of the dataprocessors configured so that it constructs a 3D image based upon thesequence of images and the tracked positions of the scanner, including:establishing, for each scanned image, transforming tensor values for atransforming tensor that expresses a relationship between a coordinatesystem defining object space and a coordinate system defining scannerspace; and transforming locations in scanner space of imaging pixelsfrom each scanned image to corresponding scanned points in object space,the transforming carried out with the transforming tensor according to:${\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime} \\1\end{bmatrix} \equiv {\begin{bmatrix}R_{11} & R_{12} & R_{13} & T_{1} \\R_{21} & R_{22} & R_{23} & T_{2} \\R_{31} & R_{32} & R_{33} & T_{3} \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}x \\y \\z \\1\end{bmatrix}}},$ wherein: the T values in the tensor express thetranslation of scanner space with respect to object space, the R valuesexpress the rotation of scanner space with respect to object space,vector x,y,z represents a location of an imaging pixel in scanner space,and vector x′,y′,z′ represents a location of scanned point in objectspace that corresponds to the location x,y,z in scanner space.
 9. Thetracker of claim 8 wherein the scanned object is an ear, and the objectspace is taken as ear space.
 10. The tracker of claim 1 wherein theimaging apparatus further comprises a probe comprising a wide-angle lensoptically coupled to an image sensor and an imaging light source, withthe lens configured so that it conducts reflected imaging illuminationfrom the scanned object to the image sensor, the image sensor isoperably coupled to at least one of the processors, and the image sensorunder processor control captures the sequence of images.
 11. The trackerof claim 1 further comprising components of the imaging apparatus thatinclude a probe that in turn includes a wide-angle lens opticallycoupled to an image sensor, and an imaging light source, with the lensconfigured so that it conducts to the image sensor imaging illuminationemitted by the imaging light source and reflected from the scannedobject, the image sensor being coupled for data communications to atleast one of the data processors, the imaging sensor capturing thesequence of images, and the at least one of the data processorsconstructs, in dependence upon the sequence of images from the sensorand the tracked positions of the scanner, a 3D image of the scannedobject.
 12. A method of tracking scanner motion, the method comprising:capturing by imaging apparatus of a scanner a sequence of images of ascanned object as the scanner moves with respect to the scanned object,the scanned object illuminated by tracking illumination from a trackingillumination emitter mounted on the scanner, sensing, by one or moretracking illumination sensors mounted on the scanner as the scannermoves in the process of scanning the scanned object, reflections oftracking illumination emitted from the tracking illumination emitter andreflected from tracking targets installed off the scanner at positionsthat are fixed relative to the scanned object; determining, by one ormore data processors coupled for data communications to the trackingillumination sensors, as the scanner moves, tracked positions of thescanner based upon values of the reflections.
 13. The method of claim 12further comprising: determining by the one or more data processors theposition of the scanner in object space when the scanner is positionedat a predetermined position with respect to the scanned object; andsetting by the one or more data processors the position of the scannerat the predetermined position as the origin of a coordinate systemdefining object space.
 14. The method of claim 12 wherein the scannedobject is an ear, and the method further comprises: determining by theone or more data processors the position of the scanner in object spacewhen the probe is positioned at the aperture of the auditory canal ofthe scanned ear; and setting by the one or more data processors theposition of the scanner at the aperture of the auditory canal of thescanned ear as the origin of a coordinate system defining object space.15. The method of claim 12 wherein the tracking targets compriseretroreflectors.
 16. The method of claim 12 wherein the trackingillumination is infrared light.
 17. The method of claim 12 furthercomprising constructing by the one or more data processors a 3D image ofthe scanned object based upon the sequence of images and the trackedpositions of the scanner.
 18. The method of claim 12 further comprisingconstructing by the one or more data processors a 3D image of thescanned object based upon the sequence of images and the trackedpositions of the scanner, with the sequence of images implemented as asequence of 2D images, including, for the sequence of 2D images from theimage sensor of the object taken when the object is illuminated byimaging light from the scanner: detecting ridge points for each 2Dimage, including identifying a set of brightest pixels for each 2Dimage, each set depicting a c-shaped broken ring of imaging lightreflecting from a surface of the scanned object; transforming, independence upon a predefined association between each pixel in the imagesensor and corresponding points in scanner space, the ridge points topoints in scanner space; and transforming, in dependence upon arelationship between an origin of a coordinate system defining scannerspace and an origin of another coordinate system defining object space,the points in scanner space to points in object space.
 19. The method ofclaim 12 further comprising constructing by the one or more dataprocessors a 3D image of the scanned object based upon the sequence ofimages and the tracked positions of the scanner, including:establishing, for each scanned image, transforming tensor values for atransforming tensor that expresses a relationship between a coordinatesystem defining object space and a coordinate system defining scannerspace; and transforming locations in scanner space of imaging pixelsfrom each scanned image to corresponding scanned points in object space,the transforming carried out with the transforming tensor according to:${\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime} \\1\end{bmatrix} \equiv {\begin{bmatrix}R_{11} & R_{12} & R_{13} & T_{1} \\R_{21} & R_{22} & R_{23} & T_{2} \\R_{31} & R_{32} & R_{33} & T_{3} \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}x \\y \\z \\1\end{bmatrix}}},$ wherein: the T values in the tensor express thetranslation of scanner space with respect to object space, the R valuesexpress the rotation of scanner space with respect to object space,vector x,y,z represents a location of an imaging pixel in scanner space,and vector x′,y′,z′ represents a location of scanned point in objectspace that corresponds to the location x,y,z in scanner space.
 20. Themethod of claim 19 wherein the scanned object is an ear, and the objectspace is taken as ear space.